Llama 3.1 8B Instruct

Llama 3.1 8B Instruct requires 32 GB of memory at 128K context due to a 16 GB KV cache, not just the 15 GB of weights, contradicting earlier reports that it fits in 24 GB. Users can reduce memory by q…

NVIDIA's Blackwell platform set a new STAC-AI benchmark record for large language model inference in financial services, outperforming prior results on the STAC-AI LANG6 test. The benchmark, developed…